Image-quality adjustment of image data

ABSTRACT

An image quality adjuster adjusts luminance values of image data sing the image data generated by an image generating device, and image generation record information associated with the image data containing at least light emission information for a supplemental light source flash at the time of generation of the image data. Specifically, a decision is made based on the light emission information for the supplemental light source as to whether to execute an image quality adjustment process; and if a decision to do so, the image data is adjusted so as to reduce the maximum luminance value, which is the largest possible luminance value the image data may have.

TECHNICAL FIELD

The present invention relates to an image adjustment technique formaking adjustments to image quality of image data.

BACKGROUND ART

Image quality of image data generated by a digital still camera (DSC),digital video camera (DVC) or the like can be adjusted as desired usingan image retouching application on a personal computer. An imageretouching application is typically provided with an image adjustingfunction for automatically adjusting image quality of image data; byusing this image adjusting function, the quality of an image output byan output device can be improved. CRTs, LCDs, printers, projectors, andtelevision receivers, for example, are known as output devices forimages.

A printer driver, which controls the operation of one of these outputdevices, namely, a printer, is also equipped with a function forautomatically adjusting image quality; image quality of printed imagescan be improved using such a printer driver as well.

However, in the automatic image quality adjusting function provided byan image retouching application or printer driver, image qualitycorrection is executed based on image data having typical imagecharacteristics. On the other hand, since the image data being subjectedto image processing can be produced under various conditions, in someinstances, image quality cannot be improved by executing a givenautomatic image quality adjusting function.

For example, where a physical object, landscape, person, video image orother photographic subject appears dark, a method of producing a flashor other supplemental light source in order to illuminate the subject isfrequently employed. If the quantity of light falling on the subjectfrom the supplemental light source is greater than the proper amount,so-called “whiteout” areas of high luminance and whitish appearance mayoccur in portions of the image obtained from the image data so produced.These whiteout areas have high luminance and tend to stand out relativeto other areas; however, it has not been intended that these whiteoutareas should stand out. When an image containing such a whiteout area issubjected to image quality correction based on image data having typicalimage characteristics, overall image quality cannot be improved in someinstances. This problem is not limited to DSCs, and is a problem commonto DVCs and other such image generating devices.

In view of the problems mentioned above, it is an object of theinvention to automatically carry out image quality adjustmentsappropriate for individual sets of image data.

DISCLOSURE OF THE INVENTION

In order to achieve the aforementioned object at least in part, an imageprocessing device according to the present invention is a device forperforming image processing using image data generated by an imagegeneration device, and image generation record information associatedwith the image data that include light emission information of asupplemental light source at the time of generation of the image data.The image processing device comprises: an image quality adjusterconfigured to decide, based on the light emission information for thesupplemental light source included in the image generation recordinformation, whether to execute an image quality adjustment process,wherein if a positive decision is made, the image quality adjusterexecutes the image quality adjustment process to adjust the image dataso as to reduce a maximum luminance value which is a largest possibleluminance value the image data may have.

The image processing device according to the invention can, on the basisof the light emission information for the supplemental light source atthe time of generation of the image data, make an appropriate decisionas to whether to execute image quality adjustment. Since image qualityadjustment is executed so as reduce the maximum luminance value, whichrepresents the largest possible luminance value the image data may take,the tendency for areas of high luminance to stand out can be inhibited.

Preferably, the image quality adjuster decides, based on the lightemission information of the supplementary light source included in theimage generation record information, whether illumination was providedby the supplemental light source at the time of generation of the imagedata, and the image quality adjuster executes the image qualityadjustment process if a judgment is made that (a) illumination wasprovided by the supplemental light source at the time of generation ofthe image data.

In this arrangement, the tendency for areas of high luminance to standout when illuminated can be inhibited.

Also preferably, the image generation record information furtherincludes information relating to a distance between a subject of theimage data and the image generation device of the image data at the timeof generation of the image data, and the image quality adjuster haltsexecution of the image quality adjustment process, or reduces amagnitude of luminance value adjustment in the image quality adjustmentprocess, irrespective of a positive or negative result of the judgment(a), if a judgment is made that (b) the distance from the subject is notwithin a first predetermined near distance range.

In this arrangement, the decision as to whether image quality adjustmentis carried out can be made appropriately on the basis of distance fromthe subject at the time that image data is generated.

The image generation record information may further include informationrelating to quantity of light of the supplemental light source at thetime of generation of the image data, and the image quality adjuster mayadjust the first predetermined near distance range based on at least thequantity of light.

In this arrangement, the decision as to whether image quality adjustmentis carried out can be made appropriately on the basis of the quantity oflight of a supplemental light source at the time that image data isgenerated.

The image generation record information may further include informationrelating to an aperture value of the image generating device at the timeof generation of the image data, and the image quality adjuster mayadjust the first predetermined near distance range based on at least theaperture value.

In this arrangement, the decision as to whether image quality adjustmentis carried out can be made appropriately on the basis of the aperturevalue of the image generating device at the time that image data isgenerated.

The image generation record information may further include informationrelating to sensitivity of an optical circuit of the image generatingdevice at the time of generation of the image data, and the imagequality adjuster may adjust the first predetermined near distance rangebased on at least the optical circuit sensitivity.

In this arrangement, the decision as to whether image quality adjustmentis carried out can be made appropriately on the basis of the sensitivityof the optical circuit of the image generating device at the time thatimage data is generated.

The image generation record information may further includes informationrelating to quantity of light of the supplemental light source at thetime of generation of the image data, and the image quality adjuster mayhalt execution of the image quality adjustment process, or reduce amagnitude of luminance value adjustment in the image quality adjustmentprocess, irrespective of a positive or negative result of the judgment(a), if a judgment is made that (c) the quantity of light is not withina second predetermined range.

In this arrangement, the decision as to whether image quality adjustmentis carried out can be made appropriately on the basis of the quantity oflight of the supplemental light source at the time that image data isgenerated.

The image quality adjuster may halt execution of the image qualityadjustment process, or reduce a magnitude of luminance value adjustmentin the image quality adjustment process, irrespective of a positive ornegative result of the judgment (a), if a judgment is made that (d) sizeof an area of interlinked pixels having luminance above a firstpredetermined luminance value in the image data is less than apredetermined threshold value.

In this arrangement, the decision as to whether image quality adjustmentis carried out can be made appropriately, on the basis of the size of anarea of interlinked pixels having luminance above a first predeterminedluminance value.

The image generation record information may further include informationrelating to quantity of light of the supplemental light source at thetime of generation of the image data, and the image quality adjuster mayadjust magnitude of luminance value adjustment in the image qualityadjustment process based on at least the quantity of light.

In this arrangement, the magnitude of luminance value adjustment can beadjusted appropriately on the basis of the quantity of light of asupplemental light source.

The image generation record information may further include informationrelating to a distance between a subject of the image data and the imagegenerating device at the time of generation of the image data, and theimage quality adjuster may adjust magnitude of luminance valueadjustment in the image quality adjustment process based on at least thedistance from the subject.

In this arrangement, the magnitude of luminance value adjustment can beadjusted appropriately on the basis of distance from a subject.

The image generation record information may further include informationrelating to an aperture value of the image generating device at the timeof generation of the image data, and the image quality adjuster mayadjust magnitude of luminance value adjustment in the image qualityadjustment process based on at least the aperture value.

In this arrangement, the magnitude of luminance value adjustment can beadjusted appropriately on the basis of aperture value.

The image generation record information may further includes informationrelating to sensitivity of an optical circuit of the image generatingdevice at the time of generation of the image data, and the imagequality adjuster may adjust magnitude of luminance value adjustment inthe image quality adjustment process based on at least the sensitivity.

In this arrangement, the magnitude of luminance value adjustment can beadjusted appropriately on the basis of optical circuit sensitivity.

The image processing device pertaining to another aspect of theinvention is an image processing device for processing image datagenerated by an image generating device, comprising an image qualityadjuster configured to execute an image quality adjustment process toadjust the image data so as to reduce a maximum luminance value which isa largest possible luminance value the image data may have, if ajudgment is made that size of an area of interlinked pixels havingluminance above a first predetermined luminance value in the image datais greater than a predetermined threshold value.

This image processing device can appropriately make a decision as towhether to execute image quality adjustment, on the basis of the size ofan area of interlinked pixels having luminance above a firstpredetermined luminance value. Additionally, since it executes imagequality adjustment so as to reduce the maximum luminance value whichrepresents the largest possible luminance value, areas of high luminancecan be prevented from standing out.

Preferably, a target area targeted for the image quality adjustmentprocess includes a first type area of interlinked maximum luminancepixels having a largest possible luminance value.

In this arrangement, areas of interlinked maximum-luminance pixels canbe prevented from standing out.

The target area targeted for the image quality adjustment process mayfurther include a second type area meeting a specific condition, andsituated in proximity to the first type area.

In this arrangement, the boundary between an area of interlinkedmaximum-luminance pixels and the surrounding area can be prevented fromstanding out.

The specific condition may include at least the condition (e) that thesecond type area is composed of pixels whose shortest distance from thefirst type area is equal to or less than a first predetermined distance.

In this arrangement, the boundary between an area of interlinkedmaximum-luminance pixels and the surrounding area can be prevented fromstanding out.

The specific condition may include at least the condition (f) that thesecond type area is composed of pixels whose luminance values are equalto or greater than a second predetermined luminance value, and is anarea interlinked with the first type area.

In this arrangement, the boundary between an area of interlinkedmaximum-luminance pixels and a surrounding area composed of pixels whoseluminance values are above a second predetermined luminance value can beprevented from standing out.

An output apparatus according to the present invention is an outputapparatus for outputting an image using image data generated by an imagegeneration device, and image generation record information associatedwith the image data that include light emission information of asupplemental light source at the time of generation of the image data.The output apparatus comprises: an image quality adjuster configured todecide, based on the light emission information for the supplementallight source included in the image generation record information,whether to execute an image quality adjustment process, wherein if apositive decision is made, the image quality adjuster executes the imagequality adjustment process to adjust the image data so as to reduce amaximum luminance value which is a largest possible luminance value theimage data may have. ; and an image output unit configured to output animage according to the quality-adjusted image data.

The invention may be realized in various embodiments, for example, animage output method and image output device; and image processing methodand image processing device; a computer program for realizing thefunctions of such a method or device; and a storage medium having such acomputer program stored thereon.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the arrangement of an image outputsystem as a embodiment of the invention.

FIG. 2 is a block diagram showing a simplified arrangement of a digitalstill camera 12.

FIG. 3 is an explanatory diagram conceptually illustrating an example ofan arrangement within an image file that can be used in this embodiment.

FIG. 4 is an explanatory diagram describing an example of data structureof an ancillary information storage field 103.

FIG. 5 is an explanatory diagram describing an example of data structurein an Exif data field.

FIG. 6 is a block diagram showing a simplified arrangement of a printer20.

FIG. 7 is a block diagram showing an arrangement of printer 20, centeredon the control circuit 30 of printer 20.

FIG. 8 is a flowchart showing a process flow for generating an imagefile GF in a digital still camera 12.

FIG. 9 is a flowchart showing a processing routine for image processingin printer 20.

FIG. 10 is a flowchart showing a processing routine for image processingbased on image generation record information.

FIG. 11 is a flowchart showing a processing routine for an automaticimage quality adjustment process.

FIG. 12 is an explanatory diagram showing the relationship betweenluminance value input level and output level in a embodiment of anautomatic image quality adjustment process.

FIG. 13 is an explanatory diagram showing the relationship between afirst predetermined value and flash intensity in a embodiment of anautomatic image quality adjustment process.

FIG. 14 is an explanatory diagram showing the relationship between afirst predetermined value and flash intensity in another embodiment ofan automatic image quality adjustment process.

FIG. 15 is an explanatory diagram showing the relationship between afirst predetermined value and F value in a embodiment of an automaticimage quality adjustment process.

FIG. 16 is an explanatory diagram showing the relationship between afirst predetermined value and ISO speed rate in a embodiment of anautomatic image quality adjustment process.

FIG. 17 is an explanatory diagram depicting an example of a whiteoutarea in an image.

FIG. 18 is an explanatory diagram depicting an example of a whiteoutarea and an image quality adjustment process target area in an image.

FIG. 19 is an explanatory diagram showing the relationship betweenluminance value input level and output level in another embodiment of anautomatic image quality adjustment process.

FIG. 20 is an explanatory diagram showing the relationship betweenluminance value input level and output level in another embodiment ofautomatic image quality adjustment processing.

FIG. 21 is an explanatory diagram showing an example of an image outputsystem in which an image data processing device may be implemented.

FIG. 22 is a flow chart showing an image processing routine from whichthe color space conversion process has been omitted.

FIG. 23 is a flowchart showing another processing routine for imageprocessing based on image generation record information.

FIG. 24 is a flowchart showing another processing routine for imageprocessing based on image generation record information.

BEST MODE FOR CARRYING OUT THE INVENTION

Output image adjustment of image files according to the presentinvention is described hereinbelow through several embodiments, in thefollowing order, with reference to the accompanying drawings.

-   A. Arrangement of Image Output System;-   B. Arrangement of Image File;-   C. Arrangement of Image Output System Capable of Using Image File;-   D. Image Processing in Digital Still Camera;-   E. Image Processing in Printer;-   F. Embodiments of Automatic Image quality Adjustment Process;-   G. Arrangement of Image Output System Employing Image Data    Processing Device;-   H. Variations    A. Arrangement of Image Output System

FIG. 1 illustrates an exemplary image output system in which an outputdevice may be implemented as a embodiment of the invention. Image outputsystem 10 comprises a digital still camera 12 as an image generatingdevice for generating image files; and a printer 20 as an image outputdevice. An image file generated by digital still camera 12 can betransferred to printer 20 via a cable CV, or by directly inserting intoprinter 20 a memory card MC on which the image file is stored. Printer20 executes image quality adjustment processing of image data based onthe image file that has been read in, and outputs an image. As theoutput device, a CRT display, LCD display, or other monitor 14, aprojector, or the like could be used in addition to or instead of aprinter 20. The following description is based on the use of a printer20 comprising an image quality adjusting module and an image outputmodule as the output device, with a memory card MC being inserteddirectly into the printer 20.

FIG. 2 is a block diagram showing a simplified arrangement of digitalstill camera 12. The digital still camera 12 of this embodimentcomprises an optical circuit 121 for collecting optical information; animage acquisition circuit 122 for controlling the optical circuit toacquire an image; an image processing circuit 123 for processing theacquired digital image; a flash 130 serving as a supplemental lightsource; and a control circuit 124 for controlling the various circuits.Control circuit 124 comprises memory, not shown. Optical circuit 121comprises a lens 125 for collecting optical information; an aperture 129for adjusting the amount of light; and a CCD 128 for converting opticaldata passing through the lens into image data.

Digital still camera 12 stores the acquired image on a memory card MC.The typical storage format for image data in digital still camera 12 isthe JPEG format, but other storage formats, such as TIFF format, GIFformat, BMP format, or RAW data format could be used instead.

Digital still camera 12 comprises a Select/Set button 126 for settingvarious shooting parameters (aperture value, flash mode, ISO speed,shooting mode, etc.); and a liquid crystal display 127. Liquid crystaldisplay 127 is used to preview a shot image, or when setting theaperture or other parameter using the Select/Set button 126.

Aperture value may be set to values within a range predetermined foreach model of digital still camera 12; for example, it may be set tocertain predetermined discrete numbers from 2 to 16 (e.g., 2, 2.8, 4,5.6 . . . etc.). F number is typically used as the aperture value.Accordingly, a larger aperture value means a smaller aperture.

Flash mode is a parameter for controlling operation of the flash 130,and may be selected from among auto flash mode, flash off mode, forcedflash mode, etc., for example.

ISO speed is a parameter that relates to the sensitivity of opticalcircuit 121. ISO speed may be set to values within a certainpredetermined range, for example, predetermined discrete values from 100to 800 (e.g., 100, 200, 400, 800 etc.). ISO speed is originally anindicator of sensitivity of silver salt film; it is used in combinationwith other parameters relating to image generation, such as aperturevalue, to set appropriate image generating parameters (shootingparameters). In a digital still camera, by using the corresponding ISOspeed as an indicator of the sensitivity of the optical circuit, imagegenerating parameters such as aperture value can be set easily.

Shooting mode may be selected from among a number of predetermined modessuch as standard mode, portrait mode, landscape mode, night scene mode,etc. Where one of these shooting modes has been indicated, the relevantparameters (aperture value, ISO speed, flash mode, etc.) are setautomatically depending on the set mode. For example, where standardmode has been selected as the shooting mode, parameters relating toimage generation are set to standard values.

Once a shot has been taken by digital still camera 12, image data andimage generation record information are stored in the form of an imagefile on memory card MC. Image generation record information can includeaperture value and other parameter settings at the time of shooting(time of generation of image data), which will be described in detailhereinbelow.

B. Arrangement of Image File

FIG. 3 is an explanatory diagram conceptually illustrating an example ofan arrangement within an image file used in this embodiment. Image fileGF comprises an image data storage field 101 for storing image data GD;and an image generation record information storage field 102 for storingimage generation record information GI. Image data GD is stored, forexample, in JPEG format, while image generation record information GI isstored, for example, in TIFF format (a format in which data and datafields are identified by tags). The terms “file structure” and “datastructure” in this embodiment refer to the structure of the file or datain the form in which the file or data etc. is stored in a memory device.

Image generation record information GI is information relating to animage when image data is generated (shot) by digital still camera 12 orother such image generating device, and includes settings such as thefollowing.

-   -   Flash (flash on/off)    -   Subject distance    -   Subject distance range    -   Flash intensity    -   Aperture value    -   ISO speed rating (ISO speed)    -   Shooting mode    -   Maker name    -   Model name    -   Gamma value

The image file GF of this embodiment may basically comprise theaforementioned image data storage field 101 and image generation recordinformation storage field 102; or may have a file structure according toan existing standardized file format. The following specific descriptionpertains to the case where the image file GF pertaining to thisembodiment conforms to the Exif file format.

An Exif file has a file structure in accordance with the digital stillcameran image file format specification (Exif); the specification hasbeen proposed by the Japan Electronics and Information TechnologyIndustries Association (JEITA). Similar to the conceptual diagram inFIG. 3, the Exif file format comprises a JPEG image data storage fieldfor storing image data in the JPEG format; and an ancillary informationstorage field for storing information of various kinds relating to thestored JPEG image data. The JPEG image data storage field corresponds tothe image data storage field 101 in FIG. 3, and the ancillaryinformation storage field to the image generation record informationstorage field 102. The ancillary information storage field stores imagegeneration record information relating to a JPEG image, such as shootingdate/time, aperture value, and subject distance.

FIG. 4 is an explanatory diagram describing an example of data structureof ancillary information storage field 103. In the Exif file format,hierarchical tags are used to designate data fields. Each data fieldcontains within it a plurality of subordinate data fields identified bysubordinate tags. In FIG. 4, areas enclosed by rectangles representsingle data fields, with tag names noted at upper left. In thisembodiment, three data fields whose tag names are APP0, APP1, and APP6are contained. The APP1 data field contains within it two data fieldswith the tag names IFD0 and IFD1. The IFD0 data field contains threedata fields with the tag names PM, Exif, and GPS. Data and data fieldsare stored according to a prescribed address or offset value; address oroffset value can be searched by means of tag name. On the output deviceend, data corresponding to desired information can be acquired by meansof specifying and address or offset value corresponding to the desiredinformation.

FIG. 5 is an explanatory diagram describing an example of data structure(data tag names and parameter values) in the Exif data field in FIG. 4,wherein tag name can be referenced by tracing in the orderAPP1-IFD0-Exif. As shown in FIG. 4, the Exif data field can include adata field with the tag name MakerNote; the MakerNote data field can inturn include a plurality of items of data, although these are omitted inFIG. 5.

As shown in FIG. 5, the Exif data field stores parameter values relatingto information such as flash, subject distance, flash intensity,aperture value, ISO speed rating, and so on. In this embodiment, “flash”and “flash intensity” are used as light emission information relating tothe quantity of light provided by a supplemental light source. Subjectdistance is used as information relating to the distance between asubject and the image generating device; aperture value is used asinformation relating to aperture value; and ISO speed rating is used asinformation relating to the sensitivity of the optical circuit. It issufficient for flash information relating to the supplemental lightsource to contain at least information for use in deciding whether ornot illumination was provided by the supplemental light source; forexample, an arrangement in which only “flash” is used as light emissioninformation for the supplemental light source is acceptable. Details ofthe decision as to whether or not illumination was provided by thesupplemental light source will be described hereinbelow.

Flash information is information relating to flash operation, and mayinclude four sets of information relating to operating mode andoperation result thereof. Operating mode may be set from among aplurality of values including the following three values, for example.

-   1: Forced flash mode-   2: Disable flash mode-   3: Auto flash mode

Operation result may be set from among two values, Flash On or FlashOff, for example. The decision as to whether or not there wasillumination by the supplemental light source during generation of imagedata may be carried out using this operation result.

Among image generating devices, certain devices are equipped with amechanism for sensing reflected light from a subject illuminated withlight from a flash. In the event that a flash cover or other obstacleblocks the flash light, or the flash fails to fire despite beingoperated, no illumination will have been provided. Such instances can beidentified through the presence or absence of reflected light. Flashinformation can include information relating to the presence/absence ofa reflected light sensing mechanism, and to the presence/absence ofreflected light sensed at the time the image data is generated (time ofshooting). In the event that a reflected light sensing mechanism ispresent and sensed reflected light is absent, it can be determined thatno illumination was provided by the supplemental light source, even ifthe aforementioned operation result was Flash On.

Subject distance information is information relating to the distancebetween the image generating device and a subject. For example, it maybe set in meter units, based on distance information set to match thefocal point at the time that image data is generated.

Flash intensity is information relating to the quantity of light emittedby the flash at the time that image data is generated; the unit ofmeasurement thereof is BCPS (Beam Candle Power Seconds), for example.

ISO speed rating information is information relating to the sensitivityof the optical circuit, and is set to a parameter value equivalent toISO speed which is an index of sensitivity of silver salt film.

Information associated with image data may be also stored appropriatelyin fields other than the Exif data field in FIG. 4. For example, makername and model name are stored in the data field whose tag name is IFD0,as information identifying the image generating device.

C. Arrangement of Image Output Device

FIG. 6 is a block diagram showing a simplified arrangement of printer 20in the embodiment. Printer 20 is a printer capable of image output, forexample, an ink jet printer that ejects ink of four colors, cyan C,magenta Mg, yellow Y, and black K, on a print medium to produce a dotpattern. An electro-photographic printer that transfers and fixes toneronto a print medium may also be used. In addition to the four colorsindicated above, light cyan LC which is lighter in density than cyan C,light magenta LM which is lighter in density than magenta Mg, and darkyellow DY which is darker in density than yellow Y, may be used as inks.Where monochromatic printing is performed, the arrangement may employblack K only; or red R or green G may be used. The type of ink or tonerused may be selected depending on the characteristics of the image foroutput.

As shown in the drawing, printer 20 comprises a mechanism for driving aprint head 211 mounted on a carriage 21, to eject ink and form dots; amechanism for reciprocating carriage 21 in the axial direction of aplaten 23 by means of a carriage motor 22; a mechanism for feedingprinter paper P by means of a paper feed motor 24; and a control circuit30. By means of these mechanisms, printer 20 functions as an imageoutput unit. The mechanism for reciprocating carriage 21 in the axialdirection of a platen 23 is composed of a slide rail 25 extendingparallel to the axis of platen 23, for slidably retaining carriage 21; apulley 27 linked via an endless drive belt 26 to carriage motor 22; anda position sensor 28 for sensing the home position of carriage 21. Themechanism for feeding printer paper P is composed of platen 23; paperfeed motor 24 which rotates platen 23; an auxiliary paper feed roller,not shown in the drawing; and a gear train (not shown) for transmittingrotation of paper feed motor 24 to platen 23 and the auxiliary paperfeed roller.

Control circuit 30 exchanges signals with a printer control panel 29while appropriately controlling operation of paper feed motor 24,carriage motor 22, and print head 211. Printer paper P supplied toprinter 20 is set between platen 23 and the auxiliary paper feed roller,and advanced by a predetermined amount depending on the rotation angleof platen 23.

Carriage 21 has a print head 211, and enables mounting of an ink jetcartridge of utilizable ink. On the bottom face of print head 211 aredisposed nozzles for ejecting utilizable ink (not shown).

FIG. 7 is a block diagram showing an arrangement of printer 20, centeredon the control circuit 30 of printer 20. Within control circuit 30 aredisposed a CPU 31; PROM 32; RAM 33; a memory card slot 34 for acquiringdata from a memory card MC; a peripheral device input/output unit (PIO)35 for exchanging data with paper feed motor 24, carriage motor 22,etc.; a drive buffer 37, and the like. Drive buffer 37 is used as abuffer for supplying dot on/off signals to print head 211. Thesecomponents are interconnected to a bus 38, enabling exchange of dataamong them. Control circuit 30 is also provided with a transmitter 39for outputting a drive waveform at predetermined frequency, and adistributed output device 40 for distributing the output of transmitter39 to print head 211 at predetermined timing.

Control circuit 30, while synchronizing with operations of paper feedmotor 24 and carriage motor 22, outputs dot data to drive buffer 37 atpredetermined timing. Control circuit 30 also reads an image file frommemory card MC, analyzes the ancillary information, and performs imageprocessing based on the image generation record information acquiredthereby. That is, control circuit 30 functions as the image qualityadjuster. The process flow of image processing executed by controlcircuit 30 will be described in detail hereinafter.

D. Image Processing in Digital Still Camera

FIG. 8 is a flowchart showing process flow for generating an image fileGF in digital still camera 12.

Control circuit 124 (FIG. 2) of digital still camera 12 generates imagedata GD in response to a shoot request, for example, depression of theshutter button (Step S100). Where aperture value, ISO speed, shootingmode and other parameter settings have been made, the set parametervalues are used when generating the image data GD.

Control circuit 124 stores the generated image data GD and imagegeneration record information GI as an image file GF on memory card MC(Step S110), and terminates the processing routine. Image generationrecord information GI includes parameters used at the time of imagegeneration, such as aperture value, ISO speed, etc.; shooting modeand/or other arbitrarily set parameter values; and parameter values setautomatically, such as maker name, model name, and the like. Image dataGD also stores as image file GF image information that has beenconverted from the RGB color space to the YCbCr color space, andcompressed in JPEG format.

By means of the aforementioned processes executed by digital stillcamera 12, both the image data GD and the image generation recordinformation GI that includes various parameter values at the time ofgeneration of image data, are constructed as an image file GF forstorage on memory card MC.

E. Image Processing in Printer

FIG. 9 is a flowchart showing a processing routine for image processingin printer 20 of the embodiment. The following description is based on ascenario wherein a memory card MC having an image file GF stored thereonis inserted directly into printer 20. When memory card MC has beeninserted into memory card slot 34, the CPU 31 of control circuit 30(FIG. 7) of printer 20 reads out the image file GF (FIG. 3) from memorycard MC (Step S200). Next, in Step S210, CPU 31 searches in theancillary information storage field of image file GF for imagegeneration record information GI indicating information at the time thatthe image data was generated. In the event that image generation recordinformation GI is found (Step S220: Y), CPU 31 acquires and analyzes theimage generation record information GI (Step S230). On the basis of theanalyzed image generation record information GI, CPU 31 executes imageprocessing, described hereinbelow (Step S240), outputs the processedimage (Step S250), and terminates the processing routine.

An image file created by a drawing application or the like, on the otherhand, will not have image generation record information GI that containsinformation such as aperture value and the like. If CPU 31 cannot findimage generation record information GI (Step S200: N), it performsstandard processing (Step S260), outputs the processed image (StepS250), and terminates the processing routine.

FIG. 10 is a flowchart showing a processing routine for image processing(corresponding to Step S240 in FIG. 9) based on the image generationrecord information. The CPU 31 of control circuit 30 (FIG. 7) of printer20 reads the image data GD from the read out image file GF (Step S300).

As mentioned previously, digital still camera 12 stores image data GD asJPEG format files, and in a JPEG format file image data is stored usingan YCbCr color space. In Step S310, CPU 31 executes an operation using3×3 matrix S to convert image data based on an YCbCr color space intoimage data based on an RGB color space. This matrix operation isrepresented by the following arithmetic expression, for example.

$\begin{matrix}{{\begin{pmatrix}R \\G \\B\end{pmatrix} = {S\begin{pmatrix}Y \\{{Cb} - 128} \\{{Cr} - 128}\end{pmatrix}}}{S = \begin{pmatrix}1 & 0 & 1.40200 \\1 & {- 0.34414} & {- 0.71414} \\1 & 1.77200 & 0\end{pmatrix}}} & \left\lbrack {{Eq}.\mspace{14mu} 1} \right\rbrack\end{matrix}$

Where the color space of image data generated by digital still camera 12is wider than a predetermined color space, for example, the sRGB colorspace, image data based on the RGB color space obtained in Step S310 mayin some instances contain useful data that is outside the defined areaof the RGB color space. Where image generation record information GIinstructs that this out-of-defined area data be handled as valid data,the out-of-defined area data will be kept as-is, while continuing withsubsequent image processing. In the absence of an instruction to handleout-of-defined area data as valid data, out-of-defined area data isclipped to the defined area. For example, where the defined area is0-255, negative value data of less than 0 is rounded to 0, and dataabove 256 to 255. In the event that the color space that is reproducibleby the image output unit is not wider than a predetermined color space,for example, the sRGB color space, it is preferable to perform clippingto the defined area regardless of any instruction in the imagegeneration record information GI. Such instances may include, forexample, cases where the image is output to a CRT, whose reproduciblecolor space is the sRGB color space.

Next, in Step S320, CPU 31 performs gamma correction and an operationemploying a matrix M, to convert image data based on an RGB color spaceto image data based on an XYZ color space. Image file GF may containgamma value and color space information at the time of image generation.In the event that image generation record information GI includes thisinformation GI, CPU 31 acquires the gamma value of the image data fromthe image generation record information GI, and executes a gammaconversion process of the image data using the acquired gamma value. CPU31 then acquires color space information for the image data from theimage generation record information GI, and executes a matrix operationof the image data using a matrix M for the color space. In the eventthat image generation record information GI does not contain a gammavalue, a gamma conversion process can be executed using a standard gammavalue. In the event that image generation record information GI does notcontain color space information, matrix operations may be executed usinga standard matrix M. A gamma value and matrix for the sRGB color spacemay be used respectively as this standard gamma value and matrix M. Thematrix operation may be given by the following arithmetic expression,for example.

$\begin{matrix}{{\begin{pmatrix}X \\Y \\Z\end{pmatrix} = {M\begin{pmatrix}{Rt}^{\prime} \\{Gt}^{\prime} \\{Bt}^{\prime}\end{pmatrix}}}{M = \begin{pmatrix}0.6067 & 0.1736 & 0.2001 \\0.2988 & 0.5868 & 0.1144 \\0 & 0.0661 & 1.1150\end{pmatrix}}{{Rt},{Gt},{{Bt} \geq 0}}{{Rt}^{\prime} = {{\left( \frac{Rt}{255} \right)^{\gamma}\mspace{14mu}{Gt}^{\prime}} = {{\left( \frac{Gt}{255} \right)^{\gamma}\mspace{14mu}{Bt}^{\prime}} = \left( \frac{Bt}{255} \right)^{\gamma}}}}{{Rt},{Gt},{{Bt} < 0}}{{Rt}^{\prime} = {{{- \left( \frac{- {Rt}}{255} \right)^{\gamma}}\mspace{14mu}{Gt}^{\prime}} = {{{- \left( \frac{- {Gt}}{255} \right)^{\gamma}}\mspace{14mu}{Bt}^{\prime}} = {- \left( \frac{- {Bt}}{255} \right)^{\gamma}}}}}} & \left\lbrack {{Eq}.\mspace{14mu} 2} \right\rbrack\end{matrix}$

The color space of image data obtained after the matrix operation hasbeen executed is an XYZ color space. The XYZ color space is an absolutecolor space, and is a device-independent color space not dependent on adevice such as a digital still camera or printer. Thus,device-independent color matching may be carried out by means of colorspace conversion to the XYZ color space.

Next, in Step S330, CPU 31 performs an operation employing a matrix N⁻¹,and inverse gamma correction to convert image data based on the XYZcolor space to image data based on wRGB color space. During inversegamma correction, CPU 31 acquires a printer-side gamma value from PROM32, and executes inverse gamma correction of the image data using theinverse of the acquired gamma value. CPU 31 then acquires from PROM 32 amatrix N⁻¹ that corresponds to the conversion from the XYZ color spaceto wRGB color space, and performs a matrix operation on the image datausing this matrix N⁻¹. This matrix operation may be given by thefollowing arithmetic expression, for example.

$\begin{matrix}{{\begin{pmatrix}{Rw} \\{Gw} \\{Bw}\end{pmatrix} = {N^{- 1}\begin{pmatrix}X \\Y \\Z\end{pmatrix}}}{N^{- 1} = \begin{pmatrix}3.30572 & {- 1.77561} & 0.73649 \\{- 1.04911} & 2.1694 & {- 1.4797} \\0.06568289 & {- 0.241078} & 1.24898\end{pmatrix}}{{Rw}^{\prime} = {{\left( \frac{Rw}{255} \right)^{1/\gamma}\mspace{14mu}{Gw}^{\prime}} = {{\left( \frac{Gw}{255} \right)^{1/\gamma}\mspace{14mu}{Bw}^{\prime}} = \left( \frac{Bw}{255} \right)^{1/\gamma}}}}} & \left\lbrack {{Eq}.\mspace{14mu} 3} \right\rbrack\end{matrix}$

Next, in Step S340, CPU 31 executes automatic adjustment processing ofimage quality. Automatic image quality adjustment processing in theembodiment involves automatic image quality adjustment processing ofimage data using the image generation record information contained inthe image file GF, especially the flash parameter values as the lightemission information for the supplemental light source. Automatic imagequality adjustment processing is described hereinbelow.

Next, in Step S350, CPU 31 executes a CMYK color conversion process anda halftone process for the purpose of printing. In the CMYK colorconversion process, CPU 31 refers to a look-up table (LUT), stored inPROM 32, for conversion from the wRGB color space to the CMYK colorspace, and converts the color space of the image data from the wRGBcolor space to the CMYK color space. That is, image data consisting ofRGB tone level values is converted to image data for use by printer 20,consisting, for example, of tone level values for six colors, C (Cyan),M (Magenta), Y (Yellow), K (Black), LC (Light Cyan), and LM (LightMagenta).

In the halftone process, CPU 31 executes a so-called halftone process toproduce halftone image data from the color-converted image data. Thishalftone image data is sequenced in the order of transmission to drivebuffer 37 (FIG. 7) to produce final print data, whereupon the processingroutine terminates. Image data processed by means of this processingroutine is output in Step S250 of the image processing routine shown inFIG. 9.

F. Embodiments of Automatic Image Quality Adjustment Process

F1. Embodiment 1 of Automatic Image Quality Adjustment Process

FIG. 11 is a flowchart showing a processing routine for automatic imagequality adjustment processing (corresponding to Step S340 in FIG. 10).CPU 31 (FIG. 7) analyzes the image generation record information GI andacquires parameter values for flash information etc. (Step S400). Next,in Step S410, CPU 31, on the basis of the acquired parameter values,makes a decision as to whether to execute image quality adjustmentprocessing (described in detail hereinbelow). In the event that it isdecided to execute image quality adjustment processing (Step S410: Y),in Step S420, CPU 31 executes image quality adjustment processing(described in detail hereinbelow). In Step S410, in the event that it isdecided not to execute image quality adjustment processing (Step S410:N), the routine terminates.

In this embodiment, CPU 31 decides to execute image quality adjustmentprocessing where the following condition is met.

(a1) Flash illumination was provided at the time of image datageneration.

The decision regarding condition (a1) is made on the basis of flashinformation parameter values. In the event that the operation resultcontained in the flash information is Flash On, CPU 31 decides thatcondition (a1) has been met. As described previously, in the event thata reflected light sensing mechanism is present and sensed reflectedlight is absent, it is decided that the condition (a1) is not met, evenif the operation result is Flash On. By so doing, image qualityadjustment processing can be executed in the event that flashillumination was provided.

FIG. 12 is an explanatory diagram showing the relationship betweenluminance value input level Yin and output level Yout in the imagequality adjustment process of this embodiment. In the embodiment, thedefined area for luminance value is 0-255. In the example shown in FIG.12, luminance values are adjusted so that where luminance value inputlevel Yin is at the maximum value within its possible range, luminancevalue output level Yout is a value smaller than the original valueYout-max. Further, the design is such that in the range over which inputlevel is smaller than a predetermined value Yin-th, luminance values arenot adjusted; and in the range over which it is larger than apredetermined value Yin-th, the proportional increase in output levelrelative to an increase in input level is smaller, the greater the inputlevel. By so doing, without changing contrast in areas where luminancevalues are smaller than the predetermined value Yin-th, luminance valuesin a whiteout area, which is white owing to high luminance, is reduced,so as to prevent the whiteout area from standing out. Printers, whichare frequently employed as image output devices, are ordinarily designedso that no ink is ejected onto areas where luminance values assume themaximum value within the possible range. Even in such instances, sinceluminance value output level is adjusted to below the maximum valuewithin the possible range, standing out of whiteout areas due to absenceof ink ejected thereon is prevented. Further, where the color space ofimage data subjected to image quality adjustment processing is a colorspace that includes brightness as a parameter, for example, where it isa YCbCr color space, image quality adjustment processing may be executedusing those luminance values. Where the color space of image data is acolor space that does not include luminance as a parameter, for example,as with the RGB color space, it is preferable to execute image qualityadjustment processing after first converting to a color space thatincludes luminance as a parameter, for example, a YCbCr color space, HSLcolor space, or the like. By so doing, adjustment of luminance valuesmay be executed easily. In this case, a process to again restore theoriginal color space is executed after execution of image qualityadjustment processing has been completed.

F2. Embodiment 2 of Automatic Image Quality Adjustment Process

In this embodiment, CPU 31 (FIG. 7) decides to execute image qualityadjustment processing when the following conditions are met.

(a1) Flash illumination was provided at the time of image datageneration; and

(b1) Subject distance is equal to or less than a first predeterminedvalue.

Condition (a1) is the same as condition (a1) in Embodiment 1. Thedecision regarding condition (b1) is executed on the basis of aparameter value of subject distance information acquired from the imagegeneration record information GI. Whiteout areas are prone to occur whenthe amount of light on the subject from a supplemental light sourceexceeds the proper amount. Also, the smaller the subject distance (i.e.the closer it is), the greater the amount of light falling on thesubject from the supplemental light source. In this embodiment, wherethe parameter value of subject distance information is equal to or lessthan a first predetermined value Lth decided in advance, for example, 2meters or less, CPU 31 (FIG. 7) decides that condition (b1) is met. Inthis way, by executing an image quality adjustment process when subjectdistance is small, whiteout areas occurring due to the subject and imagegenerating device being at near distance is prevented from standing out.In this embodiment, the range over which subject distance is equal to orless than the first predetermined value corresponds to the firstpredetermined near distance range.

F3. Embodiment 3 of Automatic Image Quality Adjustment Process

In this embodiment, as in Embodiment 2 described previously, CPU 31(FIG. 7) decides to execute an image quality adjustment process when thefollowing conditions are met.

(a1) Flash illumination was provided at the time of image datageneration; and

(b1) Subject distance is equal to or less than a first predeterminedvalue.

The difference from Embodiment 2 described previously is that the firstpredetermined value Lth for the condition (b1) used for deciding toexecute image quality adjustment processing is adjusted on the basis offlash intensity. Flash intensity is a parameter value of flash intensityinformation acquired from image generation record information GI. FIG.13 is an explanatory diagram showing relationship between firstpredetermined value Lth and flash intensity in this embodiment. Theamount of light falling on a subject from a supplemental light source isgreater the higher the flash intensity. Thus, the higher the flashintensity is, the greater will be the first predetermined value Lth,i.e. the greater (more distant) will be the threshold value for distanceat which whiteout areas are likely to occur. In the embodiment shown inFIG. 13, in consideration of the fact that the amount of light from asupplemental light source falling on the subject is inverselyproportional to the square of the subject distance, first predeterminedvalue Lth is proportional to the square root of flash intensity. CPU 31(FIG. 7) decides that condition (b1) is met when subject distance isequal to or less than a first predetermined value Lth adjusted on thebasis of flash intensity. By so doing, the decision of whether toexecute image quality adjustment processing is executed appropriately onthe basis of flash intensity. In this embodiment, the range over whichsubject distance is equal to or less than the first predetermined valuecorresponds to the first predetermined near distance range. Also, wherethe parameter value of flash intensity information is not the quantityof light value emitted by a supplemental light source, but rather avalue wherein the quantity of emitted light is calibrated for anspecific proper distance from the subject, for example, where it is aguide number, the first predetermined value Lth may be proportional toflash intensity, as shown in FIG. 14.

F4. Embodiment 4 of Automatic Image Quality Adjustment Process

In this embodiment, as in Embodiment 2 described previously, CPU 31(FIG. 7) decides to execute image quality adjustment processing wherethe following conditions are met.

(a1) Flash illumination was provided at the time of image datageneration; and

(b1) Subject distance is equal to or less than a first predeterminedvalue.

The difference from Embodiment 2 described previously is that the firstpredetermined value Lth for the condition (b1) used for deciding toexecute image quality adjustment processing is adjusted on the basis ofaperture value or F number. Aperture value or F number is a parametervalue of aperture value information acquired from image generationrecord information GI. FIG. 15 is an explanatory diagram showing therelationship between first predetermined value Lth and F number in thisembodiment. The amount of light received by optical circuit 121 (FIG. 2)of the image generating device is less the larger the F number. Thus,the larger the F number, the greater the appropriate amount of lightreceived by the subject from the supplemental light source will be. Thatis, the larger the F number, the smaller (shorter distance) will be thefirst predetermined value Lth, i.e., the distance at which whiteoutareas are prone to occur. In the embodiment shown in FIG. 15, inconsideration of the fact that the amount of light received by theoptical circuit of the image generating device is inversely proportionalto the square of the F number, and is inversely proportional to thesquare of the subject distance, the first predetermined value Lth isinversely proportional to the F number. By so doing, CPU 31 (FIG. 7) canexecute appropriately on the basis of F number the decision of whetherto execute image quality adjustment processing.

F5. Embodiment 5 of Automatic Image Quality Adjustment Process

In this embodiment, as in Embodiment 2 described previously, CPU 31(FIG. 7) decides to execute image quality adjustment processing wherethe following conditions are met.

(a1) Flash illumination was provided at the time of image datageneration; and

(b1) Subject distance is equal to or less than a first predeterminedvalue.

The difference from Embodiment 2 described previously is that the firstpredetermined value Lth for the condition (b1) used for deciding toexecute image quality adjustment processing is adjusted on the basis ofISO speed rating. ISO speed rating is a parameter value of ISO speedrating information acquired from image generation record information GI,and is a value wherein an index indicating sensitivity of opticalcircuit 121 of digital still camera 12 (FIG. 2) is represented usingcorresponding ISO speed. FIG. 16 is an explanatory diagram showing therelationship between first predetermined value Lth and ISO speed ratingin this embodiment. The luminance value of image data generated by meansof digital still camera 12 is greater the greater the ISO speed rating(sensitivity) of optical circuit 121. In other words, it may be the alsothat the apparent quantity of light received by optical circuit 121 isgreater the greater the ISO speed rating. Thus, the greater the speedrating of optical circuit 121, the smaller the appropriate quantity oflight received by a subject from the supplemental light source. That is,the greater the ISO speed rating, the greater (longer distance) thefirst predetermined value Lth, i.e., the distance at which whiteoutareas are prone to occur. In the embodiment shown in FIG. 16, inconsideration of the fact that luminance value of image data generatedby an image generating device, i.e., the apparent quantity of lightreceived by optical circuit 121, is proportional to ISO speed rating,the first predetermined value Lth is proportional to the square root ofISO speed rating. By so doing, CPU 31 (FIG. 7) can execute appropriatelyon the basis of ISO speed rating the decision of whether to executeimage quality adjustment processing.

F6. Embodiment 6 of Automatic Image Quality Adjustment Process

In this embodiment, CPU 31 (FIG. 7) decides to execute image qualityadjustment processing where the following conditions are met.

(a1) Flash illumination was provided at the time of image datageneration; and

(c1) Flash intensity is greater than a second predetermined value.

Condition (a1) is the same as condition (a1) in Embodiment 1. Thedecision regarding condition (c1) is executed on the basis of aparameter value of flash intensity information acquired from the imagegeneration record information GI. Whiteout areas occur more readilywhere the quantity of light received by a subject from the supplementallight source exceeds the appropriate quantity. In this embodiment, wherethe parameter value of flash intensity information exceeds a secondpredetermined value determined in advance, for example, 1000 BCPS, CPU31 (FIG. 7) decides that condition (c1) is met. In this way, byexecuting image quality adjustment processing when flash intensity ishigh, whiteout areas produced when the quantity of light from thesupplemental light source is too great can be prevented from standingout. In this embodiment, the range over which flash intensity is equalto or less than the second predetermined value corresponds to the secondpredetermined range.

F7. Embodiment 7 of Automatic Image Quality Adjustment Process

In this embodiment, CPU 31 (FIG. 7) decides to execute image qualityadjustment processing where the following conditions are met.

(a1) Flash illumination was provided at the time of image datageneration; and

(d1) The largest whiteout area size is greater than a predeterminedthreshold value.

Condition (a1) is the same as condition (a1) in Embodiment 1. Thedecision regarding condition (d1) is executed on the basis of the sizesof whiteout areas, obtained by analysis of image data. A whiteout areais an area of interlinked pixels having luminance values equal to orgreater than a first predetermined luminance value. As the firstpredetermined luminance value, there may be employed, for example, themaximum luminance value which is the maximum within the possible rangethereof (255 where luminance values are represented on 8 bits). In thiscase, an area in which maximum brightness pixels whose luminance valueis the maximum possible value are linked is used as a whiteout area. Avalue smaller than the maximum luminance value may be used as the firstpredetermined luminance value. By so doing, the whiteout area may bemade to include, in addition to an area of interlinked maximumbrightness pixels, the surrounding area as well. The first predeterminedluminance value may be determined on the basis of sensory evaluation ofthe output result of the image. Whiteout area size may be represented interms of the number of pixels making up the area.

As a method for deciding whether two pixels A, B are interlinked, theremay be used a method wherein it is decided that the two are interlinkedwhere pixel B is situated within a predetermined distance from pixel A.For example, in image data having pixels arrayed in grid form, wherethis predetermined distance is set to 1.5 pixels, it will be determinedthat the two pixels are linked where pixel B is situated at any of theeight-neighbor pixel locations surrounding pixel A.

As the method for dividing a plurality of pixels into pixel areas, amethod whereby they are assigned to area by treating a plurality ofpixels linked to a common pixel are treated as belonging to the samegiven area may be employed. By means of this method, one or severalwhiteout areas will be composed from a plurality of pixels havingluminance values equal to or greater than a first predeterminedluminance value.

FIG. 17 is an explanatory diagram describing an example of a whiteoutarea in an image. FIG. 17 is an image in which two fruits are shown;areas of luminance values equal to or greater than a first predeterminedluminance value are indicated by hatching. Pixels in the areas indicatedby hatching are identified as two whiteout areas 500 and 510 by theabove described area dividing method. Since whiteout areas 500 and 510are not interlinked, they are treated as separate whiteout areas. Wherea plurality of whiteout areas are present in this manner, the size ofthe largest whiteout area is used during the decision regardingcondition (d1). In the example shown in FIG. 17, the size of whiteoutarea 500 is used, for example.

In this embodiment, where whiteout area size (or where there are aplurality of whiteout zones, the largest value of size thereof isgreater than a predetermined threshold value, for example, 5% of allpixels in the image, CPU 31 (FIG. 7) decides that condition (d1) is met.By so doing, in the event that a large whiteout area should occur, thearea can be prevented from standing out.

In this embodiment, (a1) and (d1) are employed as decision conditionsfor executing image quality adjustment processing, but an arrangementwherein only (d1) is used is acceptable as well. By so doing, largewhiteout areas can be prevented from standing out regardless of whetheror not illumination is provided by a supplemental light source. Decisioncondition (d1) enables decisions to be made without using the imagegeneration record information, enabling an arrangement whereby even instandard image processing in the absence of image generation recordinformation (Step S260 in FIG. 9), image quality adjustment processingis executed in the event that decision condition (d1) is met.

F8. Embodiment 8 of Automatic Image Quality Adjustment Process

In this embodiment, in contrast to the embodiments described previously,pixels targeted for image quality adjustment processing may be selectedfrom a first type area, plus a second type area which is the areasurrounding the first type area. The first type area is a maximumluminance area of linked maximum luminance pixels whose luminance valuesare the maximum value possible. FIG. 18 is an explanatory diagramdescribing an example of a first type area and an image qualityadjustment process target area in an image. In FIG. 18, two first typeareas 600 and 610 are shown hatched; image quality adjustment processtarget areas 602, 612, which each include a first type area and asurrounding second type area, are also shown. As the second area, theremay consist of, for example, pixels whose shortest distance from thefirst type area is a first predetermined distance, for example, 5 pixelsor fewer. In this way, by targeting for processing an area that includesa first type area whose luminance value is the maximum possible and asecond type area which is the area surrounding same, image qualityadjustment processing is executed without altering image quality inareas not including the first areas, and without making the boundarybetween the first type area of maximum brightness and the surroundingarea stand out.

As a method for selecting the surrounding area as a second type area,there may be employed a method of selection on the basis of themagnitude of luminance value, rather than on the basis of only distancefrom the first type area. For example, an area composed of pixels whoseluminance values are equal to or greater than a second luminance value(which is close to maximum luminance value) and linked to the first typearea may be selected as a second type area. By so doing, in areas thatare brighter than the second luminance value, a boundary is createdbetween an area targeted for processing and an area not targeted forprocessing, so that the boundary is prevented from standing out. Thesecond luminance value may be determined on the basis of sensoryevaluation of image output results, so as to optimize image outputresults. Where luminance value adjustment is carried out as depicted inFIG. 12, the minimum luminance value Yin-th subject to adjustment ofluminance value may be used as the second predetermined luminance value.By so doing, image quality adjustment processing can be performedwithout the boundary between an area targeted for processing and an areanot targeted for processing standing out. The second predeterminedluminance value for determining an area targeted for processing may bethe same as the first predetermined luminance value for determiningwhiteout areas, used in the image quality adjustment processingdecision. In this case, whiteout areas will be selected as areastargeted for image quality adjustment processing.

F9. Embodiment 9 of Automatic Image Quality Adjustment Process

In the preceding embodiments, there is executed an image qualityadjustment process that involves adjusting the luminance value; however,an additional image quality adjustment process that involves adjustinghue and/or saturation of the area targeted for processing may beexecuted as well. For example, where there is little change in hue inthe area surrounding an area targeted for processing, the hue of thearea targeted for processing may be adjusted to the average value of huefor the surrounding area. By so doing, the hue of the area targeted forprocessing is made to approximate the hue of the surrounding area,thereby further preventing the area targeted for processing fromstanding out. As regards saturation, it is preferable to adjust it to avalue about equal to or smaller than saturation in the surrounding area.The saturation level may be represented by the average value or minimumvalue for saturation in the surrounding area. By so doing, an areatargeted for processing can be prevented from being more brightlycolored and standing out from the surrounding area.

F10. Embodiment 10 of Automatic Image Quality Adjustment Process

FIG. 19 is an explanatory diagram showing another embodiment of therelationship between luminance value input level Yin and output levelYout in an image quality adjustment process. FIG. 19 shows two exemplarycharacteristics Yout1 and Yout2. Yout1, in contrast to the embodiment inFIG. 12, is designed so that the proportion of increase in output levelrelative to an increase in input level is constant regardless of inputlevel. By so doing, change in luminance values in gradation areaswherein luminance value varies appreciably can be represented smoothly.Yout2 is designed such that there is a large rate of change in outputlevel in areas of intermediate input level. By so doing, contrast can beenhanced. The relationship between luminance value input level andoutput level may be determined on the basis of sensory evaluation ofimage output results, so as to optimize image output results. In anyevent, adjustment of luminance values is executed such that, whereluminance value input level is at maximum value within the possiblerange, luminance value output level is smaller than the original value.

F11. Embodiment 11 of Automatic Image Quality Adjustment Process

FIG. 20 is an explanatory diagram showing another embodiment of therelationship between luminance value input level and output level in animage quality adjustment process. Graph G1 is designed so that theproportion of increase in output level Yout relative to an increase ininput level Yin is greater in the area in which input level Yin is high.Thus, contrast of bright areas will be enhanced. Graphs G2 and G3 showinput/output relationships used in processing at a higher magnitude ofluminance value adjustment than in graph G1. Here, a “higher magnitudeof luminance value adjustment” means a greater proportion of increase inoutput level Yout relative to an increase in luminance value input levelYin. In preferred practice, the magnitude of luminance value adjustmentwill be higher, the greater the quantity of light received by a subject.For example, the magnitude may be designed so as to increase inassociation with an increase in flash intensity. Alternatively, themagnitude may be designed to decrease with an increase in aperture orwith an increase in subject distance, and to increase with an increasein ISO speed rating. By adjusting the magnitude of luminance valueadjustment on the basis of the image generation record information, animage quality adjustment process appropriate to image data generatedunder various different conditions can be executed. In any event,adjustment of luminance values is executed such that, where luminancevalue input level is at maximum value within the possible range,luminance value output level is smaller than the original value.

G. Arrangement of Image Output System Employing Image Data ProcessingDevice

FIG. 21 is an explanatory diagram showing an example of an image outputsystem in which an image data processing device may be implemented byway of a embodiment of the invention. Image output system 10B comprisesa digital still camera 12 as an image generating device for generatingimage files; a computer PC for executing image quality adjustmentprocessing based on an image file; and a printer 20B as an image outputdevice for outputting images. Computer PC is computer of commonly usedtype, and functions as an image data processing device. As the imageoutput device, a CRT display, LCD display, or other monitor 14B, aprojector, or the like may be used instead of printer 20B. In thefollowing description, it is assumed that a printer 20B is used as theimage output device. This embodiment differs from the image outputsystem example described previously (FIG. 1) in that the image dataprocessing device realizing the image quality adjusting unit isindependent of the image output device realizing the image output unit.The computer PC serving as the image data processing device, and theprinter realizing the image output unit, can be termed an “outputdevice” in the broad sense.

An image file created in digital still camera 12 is transferred tocomputer PC via a cable CV, or by directly inserting into computer PC amemory card MC having the image file stored thereon. Computer PCexecutes image quality adjustment processing of the image data, based onthe image file that has been read in. The image data produced by theimage quality adjustment process is transferred to printer 20B via acable CV, and is then output by printer 20B.

Computer PC comprises a CPU 150 that executes a program for realizingthe aforementioned image quality adjustment process; RAM 151 fortemporarily storing results of operations by CPU 150, image data, andthe like; and a hard disk drive (HDD) 152 for storing data needed forimage quality adjustment processing, such as an image quality adjustmentprocessing program, a lookup table, an aperture value table, and thelike. CPU 150, RAM 151, and HDD 152 function as the image qualityadjusting unit. Computer PC further comprises a memory card slot 153 forinstalling a memory card MC; and an input/output terminal 154 forconnecting a connector cable from digital still camera 12 or the like.

An image file GF generated by a digital still camera 12 is supplied tocomputer PC via a cable or via a memory card MC. When an image dataprocessing application program, either an image retouching applicationprogram or a printer driver, is run under user control, CPU 150 executesan image processing routine (FIG. 9) to process the image file GF whichhas been read in. Alternatively, the image data processing applicationprogram may be set to start up automatically when a memory card MC isinserted into memory card slot 153, or when connection of a digitalstill camera 12 to input/output terminal 154 via a cable is detected.

Image data processed by CPU 150, rather than being output in Step S250of the image processing routine (FIG. 9), is instead transferred to animage output device, for example, printer 20B, whereupon the imageoutput device receiving the image data executes image output.

In this embodiment, since image processing is carried out using an imagequality adjusting unit provided to computer PC, it is possible to use animage output device that does not have an image quality adjusting unit.Where the image output device per se comprises an image qualityadjusting unit, image data may be output to the image output devicewithout being subjected to image processing on computer PC, with imageprocessing instead being carried out by the image quality adjusting unitof the image output device. Alternatively, where the image generatingdevice comprises an image quality adjusting unit (for example, wheredigital still camera 12 comprises an image quality adjusting unit), animage generating device arrangement whereby image processing isperformed within the image generating device, and the processed imagedata then output directly to a printer, monitor or other image outputdevice may also be possible.

As described hereinabove in the embodiments, image quality of an imagein which a whiteout area would occur can be adjusted automatically, sothat output results of high quality can be obtained readily.

The invention is not limited to the embodiments set forth hereinabove,and may be reduced to practice in various embodiments without departingfrom the scope and spirit thereof. The following variations arepossible, for example.

H. Variations

H1. Variation 1

In the embodiments hereinabove, the first predetermined value for thedecision condition (b1) for executing an image quality adjustmentprocess may be designed to be adjustable on the basis of a plurality ofparameter values. For example, an arrangement in which adjustment ismade on the basis of flash intensity and aperture value or F number ispossible. An arrangement in which a parameter value for use in adjustingthe first predetermined value is selected based on types of informationincluded in image generation record information GI may be made as well.For example, where image generation record information GI includes flashintensity and ISO speed rating, there is possible an arrangement wherebythe first predetermined value is adjusted on the basis of flashintensity and ISO speed rating; or where flash intensity and aperturevalue are included, the first predetermined value may be adjusted on thebasis of flash intensity and aperture value. In this way, by selectingdecision conditions based on information included in image generationrecord information GI, more appropriate decisions can be executed.

H2. Variation 2:

An arrangement whereby a decision condition for executing image qualityadjustment processing is selected for use among a plurality of decisionconditions based on information included in image generation recordinformation GI is also possible. For example, where the image generationrecord information GI includes flash information, subject distanceinformation, and aperture value information, the decision may beexecuted on the basis of decision conditions (a1), (b1), and (d1); wherethe image generation record information GI includes only flashinformation, the decision may be executed on the basis of decisionconditions (a1) and (d1). In this way, by selecting a decision conditionbased on information included in image generation record information GI,more appropriate decisions can be executed.

H3. Variation 3:

As information relating to the distance between a subject and the imagegenerating device, employed as the aforementioned decision condition(b1), distance information other than subject distance information butthat enables a distance range to be set as a parameter value may be usedinstead. For example, the decision regarding condition (b1) may be madeusing subject distance range information set by selecting from amongthree distance ranges, macro (0-1 meter), close-up (1-3 meters), orlandscape (3 meters or more). In this case, the decision regardingcondition (b1) is made by pre-establishing a typical distance for eachof the distance ranges, and comparing the typical distance with a firstpredetermined value of condition (b1). As a typical distance, forexample, for a distance range having an upper limit value and a lowerlimit value established for distance, the median value may be used; andfor a distance range defined only by an upper limit value or by a lowerlimit value, the upper or lower limit value may be used. Similarly, themagnitude of luminance value adjustment may be adjusted on the basis ofsubject distance range.

H4. Variation 4:

Where generation of image data is performed with a supplemental lightsource situated apart from an image generating device, it is preferableto execute the decision regarding execution of an image qualityadjustment process on the basis of information relating to the distancebetween the supplemental light source and the subject. As a method fordeciding based on the distance between a supplemental light source and asubject, there may be used a method whereby, for example, in condition(b1) in the preceding embodiments, the decision is made using thedistance between supplemental light source and the subject, rather thansubject distance. By so doing, a whiteout area produced as a result ofthe short distance between a supplemental light source and a subject canbe prevented from standing out. Similarly, the magnitude of luminancevalue adjustment may adjusted on the basis of distance between asupplemental light source and the subject.

H5. Variation 5:

Where an image file GF does not include image data gamma value or colorspace information, the color space conversion process of the imageprocessing routine shown in FIG. 10 (Step S320 and Step S330) may beomitted. FIG. 22 is a flow chart showing an image processing routinefrom which the color space conversion process has been omitted. Imagedata acquired in Step S500 is converted from image data based on a YCbCrcolor space to data based on an RGB color space in Step S510. Next, inStep S520, automatic image quality adjustment processing is executedusing the image data obtained in Step S510. Next, in Step S530, a CYMKconversion process and halftone process for printing are executed.

H6. Variation 6:

In the preceding embodiments, automatic image quality adjustmentprocessing is executed after executing color space conversion; however,color space conversion may instead be executed after executing automaticimage quality adjustment processing. For example, image processing maybe executed according to the flowchart shown in FIG. 23.

H7. Variation 7:

In the preceding embodiments, a printer is used as the image outputunit, but an image output unit other than a printer may be used instead.FIG. 24 is a flowchart showing a processing routine for image processingbased on image generation record information where a CRT is used as theimage output unit. In contrast to the flowchart in FIG. 10 which depictsthe use of a printer as the image output unit, the CYMK conversionprocess and halftone process for printing are omitted. Also, since a CRTcan represent an RGB color space of image data obtained by executing amatrix operation (S), the color space conversion process is omitted aswell. Where image data based on an RGB color space obtained in Step S610includes data outside the defined area of the RGB color space, theout-of-defined-area data is clipped, and step S620 is then executed.Where the color space utilizable by an image output unit is differentfrom an RGB color space, a color conversion process to a color spaceutilizable by the image output unit is executed in a manner analogous toexecuting a CMYK color conversion process where a printer is used, andthe resultant image is output by the image output unit.

H8. Variation 8:

In the preceding embodiments, description was made taking an Exif formatfile as a specific example of the image file GF; however, the format ofthe image file pertaining to the present invention is not limited tothis. That is, any image file containing image data generated by animage generating device, and image generation record information GIdescribing conditions (information) at the time of generation of theimage data, is acceptable. With such a file, image quality of image datagenerated by an image generating device may be appropriately adjustedautomatically and output from an output device.

H9. Variation 9:

Values of matrices S, N⁻¹, and M in the equations are merely exemplary,and may be modified appropriately depending on color space on which theimage file is based, color space utilizable by an image output unit, orthe like.

H10. Variation 10:

In the preceding embodiments, use of a digital still camera 12 as theimage generating device was described; however, image files may begenerated using a different image generating device, such as a scanner,digital video camera, or the like instead.

H11. Variation 11:

In the preceding embodiments, the description took the example of a casewhere image data GD and image generation record information GI arecontained in the same image file GF, but image data GD and imagegeneration record information GI need not necessarily be stored withinthe same file. That is, it is sufficient for image data GD and imagegeneration record information GI to be associated with each other; forexample, it is acceptable to generate associating data that associatesimage data GD with image generation record information GI; store one orseveral sets of image data and image generation record information GI inindependent files; and refer to the associated image generation recordinformation GI when processing the image data GD. This is because, inthis case, although the image data GD and image generation recordinformation GI are stored in separate files, at the point in time ofimage processing which utilizes the image generation record informationGI, the image data GD and image generation record information GI are ininseparably linked, and thus function substantially the same as if theywere stored in the same file. That is, the term “image file GF” in thepresent embodiment includes those of a form wherein image data GD andimage generation record information GI are associated, at least at thepoint in time that image processing takes place. The image file GF alsoincludes motion video files stored on optical disk media such as CD-ROM,CD-R, DVD-ROM, DVD-RAM, and the like.

INDUSTRIAL APPLICABILITY

The present invention is applicable to various image processing devicesincluding computers, printers, facsimile machines, and digital cameras.

1. An image processing method for performing image processing usingimage data generated by an image generation device, and image generationrecord information associated with the image data that includes lightemission information of a supplemental light source at a time ofgeneration of the image data and information relating to a distancebetween a subject of the image data and the image generation device ofthe image data at the time of generation of the image data, the methodcomprising: (i) deciding, based on the light emission information forthe supplemental light source included in the image generation recordinformation, whether to execute an image quality adjustment process; and(ii) if a positive decision is made, executing the image qualityadjustment process to adjust the image data so as to reduce a maximumluminance value which is a largest possible luminance value the imagedata may have, wherein the deciding whether to execute the image qualityadjustment process includes deciding, based on the light emissioninformation of the supplemental light source included in the imagegeneration record information, whether illumination was provided by thesupplemental light source at the time of generation of the image data,the executing of the image quality adjustment process includes executingthe image quality adjustment process if a judgment is made that (a)illumination was provided by the supplemental light source at the timeof generation of the image data, the executing of the image qualityadjustment process includes halting execution of the image qualityadjustment process, or reducing a magnitude of luminance valueadjustment in the image quality adjustment process, irrespective of apositive or negative result of the judgment (a), if a judgment is madethat (b) the distance from the subject is not within a first neardistance range, the image generation record information further includesinformation relating to quantity of light of the supplemental lightsource at the time of generation of the image data, and the executing ofthe image quality adjustment process includes adjusting the first neardistance range to be a range nearer than a range in which thesupplemental light source is capable of illuminating a subject with thequantity of light, based on at least the quantity of light.
 2. An imageprocessing device for performing image processing using image datagenerated by an image generation device, and image generation recordinformation associated with the image data that includes light emissioninformation of a supplemental light source at a time of generation ofthe image data and information relating to a distance between a subjectof the image data and the image generation device of the image data atthe time of generation of the image data, the image processing devicecomprising: an image quality adjuster configured to decide, based on thelight emission information for the supplemental light source included inthe image generation record information, whether to execute an imagequality adjustment process, wherein if a positive decision is made, theimage quality adjuster executes the image quality adjustment process toadjust the image data so as to reduce a maximum luminance value which isa largest possible luminance value the image data may have, wherein indeciding whether to execute the image quality adjustment process, theimage quality adjuster is further configured to decide, based on thelight emission information of the supplemental light source included inthe image generation record information, whether illumination wasprovided by the supplemental light source at the time of generation ofthe image data, the image quality adjuster executes the image qualityadjustment process if a judgment is made that (a) illumination wasprovided by the supplemental light source at the time of generation ofthe image data, the image quality adjuster halts execution of the imagequality adjustment process, or reduces a magnitude of luminance valueadjustment in the image quality adjustment process, irrespective of apositive or negative result of the judgment (a), if a judgment is madethat (b) the distance from the subject is not within a first neardistance range, the image generation record information further includesinformation relating to quantity of light of the supplemental lightsource at the time of generation of the image data, and the imagequality adjuster executes the image quality adjustment process to adjustthe first near distance range to be a range nearer than a range in whichthe supplemental light source is capable of illuminating a subject withthe quantity of light, based on at least the quantity of light.
 3. Acomputer program product for causing a computer to perform imageprocessing using image data generated by an image generation device, andimage generation record information associated with the image data thatincludes light emission information of a supplemental light source at atime of generation of the image data and information relating to adistance between a subject of the image data and the image generationdevice of the image data at the time of generation of the image data,comprising: a computer readable storage medium; and a computer programstored on the computer readable storage medium, the computer programincluding: a first program for causing a computer to decide, based onthe light emission information for the supplemental light sourceincluded in the image generation record information, whether to executean image quality adjustment process; and a second program for causingthe computer, if a positive decision is made, to execute the imagequality adjustment process to adjust the image data so as to reduce amaximum luminance value which is a largest possible luminance value theimage data may have, wherein the first program for causing the computerto decide whether to execute the image quality adjustment processincludes causing the computer to decide, based on the light emissioninformation of the supplemental light source included in the imagegeneration record information, whether illumination was provided by thesupplemental light source at the time of generation of the image data,the second program for causing the computer to execute the image qualityadjustment process includes causing the computer to execute the imagequality adjustment process if a judgment is made that (a) illuminationwas provided by the supplemental light source at the time of generationof the image data, the second program for causing the computer toexecute the image quality adjustment process includes causing thecomputer to execute the image quality adjustment process, or reduce amagnitude of luminance value adjustment in the image quality adjustmentprocess, irrespective of a positive or negative result of the judgment(a), if a judgment is made that (b) the distance from the subject is notwithin a first near distance range, the image generation recordinformation further includes information relating to quantity of lightof the supplemental light source at the time of generation of the imagedata, and the second program for causing the computer to execute theimage quality adjustment process includes causing the computer to adjustthe first near distance range to be a range nearer than a range in whichthe supplemental light source is capable of illuminating a subject withthe quantity of light, based on at least the quantity of light.